2000 character limit reached
Continual learning for surface defect segmentation by subnetwork creation and selection
Published 8 Dec 2023 in cs.CV and cs.LG | (2312.05100v1)
Abstract: We introduce a new continual (or lifelong) learning algorithm called LDA-CP&S that performs segmentation tasks without undergoing catastrophic forgetting. The method is applied to two different surface defect segmentation problems that are learned incrementally, i.e. providing data about one type of defect at a time, while still being capable of predicting every defect that was seen previously. Our method creates a defect-related subnetwork for each defect type via iterative pruning and trains a classifier based on linear discriminant analysis (LDA). At the inference stage, we first predict the defect type with LDA and then predict the surface defects using the selected subnetwork. We compare our method with other continual learning methods showing a significant improvement -- mean Intersection over Union better by a factor of two when compared to existing methods on both datasets. Importantly, our approach shows comparable results with joint training when all the training data (all defects) are seen simultaneously
- Ojala, T., Pietikainen, M., Maenpaa, T.: Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. IEEE Transactions on pattern analysis and machine intelligence 24(7), 971–987 (2002) Chao and Tsai [2008] Chao, S.-M., Tsai, D.-M.: An anisotropic diffusion-based defect detection for low-contrast glass substrates. Image and Vision Computing 26(2), 187–200 (2008) Song and Yan [2013] Song, K., Yan, Y.: A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science 285, 858–864 (2013) Jeon et al. [2014] Jeon, Y.-J., Choi, D.-c., Lee, S.J., Yun, J.P., Kim, S.W.: Defect detection for corner cracks in steel billets using a wavelet reconstruction method. JOSA A 31(2), 227–237 (2014) Jia et al. [2004] Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Chao, S.-M., Tsai, D.-M.: An anisotropic diffusion-based defect detection for low-contrast glass substrates. Image and Vision Computing 26(2), 187–200 (2008) Song and Yan [2013] Song, K., Yan, Y.: A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science 285, 858–864 (2013) Jeon et al. [2014] Jeon, Y.-J., Choi, D.-c., Lee, S.J., Yun, J.P., Kim, S.W.: Defect detection for corner cracks in steel billets using a wavelet reconstruction method. JOSA A 31(2), 227–237 (2014) Jia et al. [2004] Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, K., Yan, Y.: A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science 285, 858–864 (2013) Jeon et al. [2014] Jeon, Y.-J., Choi, D.-c., Lee, S.J., Yun, J.P., Kim, S.W.: Defect detection for corner cracks in steel billets using a wavelet reconstruction method. JOSA A 31(2), 227–237 (2014) Jia et al. [2004] Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Jeon, Y.-J., Choi, D.-c., Lee, S.J., Yun, J.P., Kim, S.W.: Defect detection for corner cracks in steel billets using a wavelet reconstruction method. JOSA A 31(2), 227–237 (2014) Jia et al. [2004] Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Chao, S.-M., Tsai, D.-M.: An anisotropic diffusion-based defect detection for low-contrast glass substrates. Image and Vision Computing 26(2), 187–200 (2008) Song and Yan [2013] Song, K., Yan, Y.: A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science 285, 858–864 (2013) Jeon et al. [2014] Jeon, Y.-J., Choi, D.-c., Lee, S.J., Yun, J.P., Kim, S.W.: Defect detection for corner cracks in steel billets using a wavelet reconstruction method. JOSA A 31(2), 227–237 (2014) Jia et al. [2004] Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, K., Yan, Y.: A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science 285, 858–864 (2013) Jeon et al. [2014] Jeon, Y.-J., Choi, D.-c., Lee, S.J., Yun, J.P., Kim, S.W.: Defect detection for corner cracks in steel billets using a wavelet reconstruction method. JOSA A 31(2), 227–237 (2014) Jia et al. [2004] Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Jeon, Y.-J., Choi, D.-c., Lee, S.J., Yun, J.P., Kim, S.W.: Defect detection for corner cracks in steel billets using a wavelet reconstruction method. JOSA A 31(2), 227–237 (2014) Jia et al. [2004] Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Song, K., Yan, Y.: A noise robust method based on completed local binary patterns for hot-rolled steel strip surface defects. Applied Surface Science 285, 858–864 (2013) Jeon et al. [2014] Jeon, Y.-J., Choi, D.-c., Lee, S.J., Yun, J.P., Kim, S.W.: Defect detection for corner cracks in steel billets using a wavelet reconstruction method. JOSA A 31(2), 227–237 (2014) Jia et al. [2004] Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Jeon, Y.-J., Choi, D.-c., Lee, S.J., Yun, J.P., Kim, S.W.: Defect detection for corner cracks in steel billets using a wavelet reconstruction method. JOSA A 31(2), 227–237 (2014) Jia et al. [2004] Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Jeon, Y.-J., Choi, D.-c., Lee, S.J., Yun, J.P., Kim, S.W.: Defect detection for corner cracks in steel billets using a wavelet reconstruction method. JOSA A 31(2), 227–237 (2014) Jia et al. [2004] Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Jia, H., Murphey, Y.L., Shi, J., Chang, T.-S.: An intelligent real-time vision system for surface defect detection. In: Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004., vol. 3, pp. 239–242 (2004). IEEE Agarwal et al. [2011] Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Agarwal, K., Shivpuri, R., Zhu, Y., Chang, T.-S., Huang, H.: Process knowledge based multi-class support vector classification (pk-msvm) approach for surface defects in hot rolling. Expert Systems with Applications 38(6), 7251–7262 (2011) Shanmugamani et al. [2015] Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Shanmugamani, R., Sadique, M., Ramamoorthy, B.: Detection and classification of surface defects of gun barrels using computer vision and machine learning. Measurement 60, 222–230 (2015) He et al. [2019] He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- He, Y., Song, K., Meng, Q., Yan, Y.: An end-to-end steel surface defect detection approach via fusing multiple hierarchical features. IEEE transactions on instrumentation and measurement 69(4), 1493–1504 (2019) Tabernik et al. [2020] Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Tabernik, D., Šela, S., Skvarč, J., Skočaj, D.: Segmentation-based deep-learning approach for surface-defect detection. Journal of Intelligent Manufacturing 31(3), 759–776 (2020) Aslam et al. [2020] Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Aslam, M., Khan, T.M., Naqvi, S.S., Holmes, G., Naffa, R.: Ensemble convolutional neural networks with knowledge transfer for leather defect classification in industrial settings. IEEE Access 8, 198600–198614 (2020) Wu and Lv [2021] Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Wu, H., Lv, Q.: Hot-rolled steel strip surface inspection based on transfer learning model. Journal of Sensors 2021, 1–8 (2021) Ronneberger et al. [2015] Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241 (2015). Springer He et al. [2019] He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- He, D., Xu, K., Zhou, P.: Defect detection of hot rolled steels with a new object detection framework called classification priority network. Computers & Industrial Engineering 128, 290–297 (2019) Song et al. [2020] Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Song, G., Song, K., Yan, Y.: Edrnet: Encoder–decoder residual network for salient object detection of strip steel surface defects. IEEE Transactions on Instrumentation and Measurement 69(12), 9709–9719 (2020) Hao et al. [2021] Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Hao, R., Lu, B., Cheng, Y., Li, X., Huang, B.: A steel surface defect inspection approach towards smart industrial monitoring. Journal of Intelligent Manufacturing 32, 1833–1843 (2021) Huang et al. [2020] Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Huang, Y., Qiu, C., Yuan, K.: Surface defect saliency of magnetic tile. The Visual Computer 36, 85–96 (2020) Vaswani et al. [2017] Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. Advances in neural information processing systems 30 (2017) Pan and Zhang [2022] Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Pan, Y., Zhang, L.: Dual attention deep learning network for automatic steel surface defect segmentation. Computer-Aided Civil and Infrastructure Engineering 37(11), 1468–1487 (2022) Üzen et al. [2022] Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Üzen, H., Türkoğlu, M., Yanikoglu, B., Hanbay, D.: Swin-mfinet: Swin transformer based multi-feature integration network for detection of pixel-level surface defects. Expert Systems with Applications 209, 118269 (2022) Lv et al. [2020] Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Lv, X., Duan, F., Jiang, J.-j., Fu, X., Gan, L.: Deep metallic surface defect detection: The new benchmark and detection network. Sensors 20(6), 1562 (2020) Feng et al. [2021] Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Feng, X., Gao, X., Luo, L.: X-sdd: A new benchmark for hot rolled steel strip surface defects detection. Symmetry 13(4), 706 (2021) Liu and Ye [2022] Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Liu, T., Ye, W.: A semi-supervised learning method for surface defect classification of magnetic tiles. Machine Vision and Applications 33(2), 35 (2022) French [1999] French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- French, R.M.: Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3(4), 128–135 (1999) Goodfellow et al. [2013] Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Goodfellow, I.J., Mirza, M., Xiao, D., Courville, A., Bengio, Y.: An empirical investigation of catastrophic forgetting in gradient-based neural networks. arXiv preprint arXiv:1312.6211 (2013) Coop et al. [2013] Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Coop, R., Mishtal, A., Arel, I.: Ensemble learning in fixed expansion layer networks for mitigating catastrophic forgetting. IEEE transactions on neural networks and learning systems 24(10), 1623–1634 (2013) De Lange et al. [2021] De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- De Lange, M., Aljundi, R., Masana, M., Parisot, S., Jia, X., Leonardis, A., Slabaugh, G., Tuytelaars, T.: A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence 44(7), 3366–3385 (2021) Song et al. [2020] Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Song, G., Song, K., Yan, Y.: Saliency detection for strip steel surface defects using multiple constraints and improved texture features. Optics and Lasers in Engineering 128, 106000 (2020) Thrun [1998] Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Thrun, S.: Lifelong learning algorithms. In: Learning to Learn, pp. 181–209. Springer, ??? (1998) Li and Hoiem [2017] Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Li, Z., Hoiem, D.: Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence 40(12), 2935–2947 (2017) Zenke et al. [2017] Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Zenke, F., Poole, B., Ganguli, S.: Continual learning through synaptic intelligence. In: International Conference on Machine Learning, pp. 3987–3995 (2017). PMLR Aljundi et al. [2018] Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Aljundi, R., Babiloni, F., Elhoseiny, M., Rohrbach, M., Tuytelaars, T.: Memory aware synapses: Learning what (not) to forget. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 139–154 (2018) Rebuffi et al. [2017] Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Rebuffi, S.-A., Kolesnikov, A., Sperl, G., Lampert, C.H.: icarl: Incremental classifier and representation learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2001–2010 (2017) Castro et al. [2018] Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Castro, F.M., Marín-Jiménez, M.J., Guil, N., Schmid, C., Alahari, K.: End-to-end incremental learning. In: Proceedings of the European Conference on Computer Vision (ECCV), pp. 233–248 (2018) Douillard et al. [2020] Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Douillard, A., Cord, M., Ollion, C., Robert, T., Valle, E.: Podnet: Pooled outputs distillation for small-tasks incremental learning. In: Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX 16, pp. 86–102 (2020). Springer Mallya and Lazebnik [2018] Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Mallya, A., Lazebnik, S.: Packnet: Adding multiple tasks to a single network by iterative pruning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7765–7773 (2018) Sokar et al. [2022] Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Sokar, G., Mocanu, D.C., Pechenizkiy, M.: Avoiding forgetting and allowing forward transfer in continual learning via sparse networks. In: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pp. 85–101 (2022). Springer Dekhovich et al. [2023] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Continual prune-and-select: class-incremental learning with specialized subnetworks. Applied Intelligence, 1–16 (2023) Yan et al. [2021] Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Yan, S., Xie, J., He, X.: Der: Dynamically expandable representation for class incremental learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3014–3023 (2021) Wang et al. [2022] Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Wang, F.-Y., Zhou, D.-W., Ye, H.-J., Zhan, D.-C.: Foster: Feature boosting and compression for class-incremental learning. In: European Conference on Computer Vision, pp. 398–414 (2022). Springer Kirkpatrick et al. [2017] Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A., et al.: Overcoming catastrophic forgetting in neural networks. Proceedings of the national academy of sciences 114(13), 3521–3526 (2017) Zhang et al. [2020] Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Zhang, J., Zhang, J., Ghosh, S., Li, D., Tasci, S., Heck, L., Zhang, H., Kuo, C.-C.J.: Class-incremental learning via deep model consolidation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 1131–1140 (2020) Yoon et al. [2017] Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Yoon, J., Yang, E., Lee, J., Hwang, S.J.: Lifelong learning with dynamically expandable networks. arXiv preprint arXiv:1708.01547 (2017) Wortsman et al. [2020] Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Wortsman, M., Ramanujan, V., Liu, R., Kembhavi, A., Rastegari, M., Yosinski, J., Farhadi, A.: Supermasks in superposition. Advances in Neural Information Processing Systems 33, 15173–15184 (2020) Baweja et al. [2018] Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Baweja, C., Glocker, B., Kamnitsas, K.: Towards continual learning in medical imaging. In: Medical Imaging Meets NIPS Workshop, 32nd Conference on Neural Information Processing Systems (NIPS) (2018) van Garderen et al. [2019] Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Garderen, K., Voort, S., Incekara, F., Smits, M., Klein, S.: Towards continuous learning for glioma segmentation with elastic weight consolidation. In: International Conference on Medical Imaging with Deep Learning –Extended Abstract Track (2019) Klingner et al. [2020] Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Klingner, M., Bär, A., Donn, P., Fingscheidt, T.: Class-incremental learning for semantic segmentation re-using neither old data nor old labels. In: 2020 IEEE 23rd International Conference on Intelligent Transportation Systems (ITSC), pp. 1–8 (2020). IEEE Douillard et al. [2021] Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Douillard, A., Chen, Y., Dapogny, A., Cord, M.: Plop: Learning without forgetting for continual semantic segmentation. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4040–4050 (2021) Yan et al. [2021] Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Yan, S., Zhou, J., Xie, J., Zhang, S., He, X.: An em framework for online incremental learning of semantic segmentation. In: Proceedings of the 29th ACM International Conference on Multimedia, pp. 3052–3060 (2021) Cha et al. [2021] Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Cha, S., Yoo, Y., Moon, T., et al.: Ssul: Semantic segmentation with unknown label for exemplar-based class-incremental learning. Advances in neural information processing systems 34, 10919–10930 (2021) Qiu et al. [2023] Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Qiu, Y., Shen, Y., Sun, Z., Zheng, Y., Chang, X., Zheng, W., Wang, R.: Sats: Self-attention transfer for continual semantic segmentation. Pattern Recognition 138, 109383 (2023) Tercan et al. [2022] Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Tercan, H., Deibert, P., Meisen, T.: Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer. Journal of Intelligent Manufacturing 33(1), 283–292 (2022) Maschler et al. [2021] Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Maschler, B., Pham, T.T.H., Weyrich, M.: Regularization-based continual learning for anomaly detection in discrete manufacturing. Procedia CIRP 104, 452–457 (2021) Maschler et al. [2022] Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Maschler, B., Tatiyosyan, S., Weyrich, M.: Regularization-based continual learning for fault prediction in lithium-ion batteries. Procedia CIRP 112, 513–518 (2022) Sun et al. [2022] Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Sun, W., Kontar, R.A., Jin, J., Chang, T.-S.: A continual learning framework for adaptive defect classification and inspection. arXiv preprint arXiv:2203.08796 (2022) Kim et al. [2020] Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Kim, E.S., Kim, J.U., Lee, S., Moon, S.-K., Ro, Y.M.: Class incremental learning with task-selection. In: 2020 IEEE International Conference on Image Processing (ICIP), pp. 1846–1850 (2020). IEEE Kim et al. [2022] Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Kim, G., Xiao, C., Konishi, T., Ke, Z., Liu, B.: A theoretical study on solving continual learning. Advances in Neural Information Processing Systems 35, 5065–5079 (2022) Dorfer et al. [2015] Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Dorfer, M., Kelz, R., Widmer, G.: Deep linear discriminant analysis. arXiv preprint arXiv:1511.04707 (2015) Hayes and Kanan [2020] Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Hayes, T.L., Kanan, C.: Lifelong machine learning with deep streaming linear discriminant analysis. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, pp. 220–221 (2020) Tan and Le [2019] Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Tan, M., Le, Q.: Efficientnet: Rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning, pp. 6105–6114 (2019). PMLR Deng et al. [2009] Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Deng, J., Dong, W., Socher, R., Li, L.-J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009). Ieee Dekhovich et al. [2021] Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Dekhovich, A., Tax, D.M., Sluiter, M.H., Bessa, M.A.: Neural network relief: a pruning algorithm based on neural activity. arXiv preprint arXiv:2109.10795 (2021) Han et al. [2015] Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Han, S., Pool, J., Tran, J., Dally, W.: Learning both weights and connections for efficient neural network. Advances in neural information processing systems 28 (2015) Li et al. [2016] Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Li, H., Kadav, A., Durdanovic, I., Samet, H., Graf, H.P.: Pruning filters for efficient convnets. arXiv preprint arXiv:1608.08710 (2016) Lee et al. [2019] Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Lee, N., Ajanthan, T., Torr, P.H.: Snip: Single-shot network pruning based on connection sensitivity. In: International Conference on Learning Representations (ICLR) (2019) Dasgupta and Hsu [2007] Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Dasgupta, S., Hsu, D.: On-line estimation with the multivariate gaussian distribution. In: International Conference on Computational Learning Theory, pp. 278–292 (2007). Springer Rajasegaran et al. [2020] Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Rajasegaran, J., Khan, S., Hayat, M., Khan, F.S., Shah, M.: itaml: An incremental task-agnostic meta-learning approach. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 13588–13597 (2020) Salehi et al. [2017] Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Salehi, S.S.M., Erdogmus, D., Gholipour, A.: Tversky loss function for image segmentation using 3d fully convolutional deep networks. In: International Workshop on Machine Learning in Medical Imaging, pp. 379–387 (2017). Springer Lin et al. [2017] Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Lin, T.-Y., Goyal, P., Girshick, R., He, K., Dollár, P.: Focal loss for dense object detection. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2980–2988 (2017) Kingma and Ba [2014] Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014) Masana et al. [2020] Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020) Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
- Masana, M., Twardowski, B., Weijer, J.: On class orderings for incremental learning. arXiv preprint arXiv:2007.02145 (2020)
Paper Prompts
Sign up for free to create and run prompts on this paper using GPT-5.
Top Community Prompts
Collections
Sign up for free to add this paper to one or more collections.